class: center, middle, inverse, title-slide .title[ # Simple linear regression ] .subtitle[ ## Lecture 1 ] .author[ ### Manuel Villarreal ] .date[ ### 08/26/24 ] --- ### Blood pressure - Researchers at UT Austin want to study the relationship between systolic blood pressure and age. They design a study to measure the blood pressure of 200 individuals alongside other variables of interest. -- - Let's first look at the data. --- ### Data: blood pressure <div class="datatables html-widget html-fill-item" id="htmlwidget-72a744e1a357b3e52df6" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-72a744e1a357b3e52df6">{"x":{"filter":"none","vertical":false,"data":[[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200],["female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","female","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male","male"],[49,52,47,46,47,45,50,62,44,40,46,47,46,48,49,45,57,60,52,47,51,46,48,47,46,62,52,51,51,48,44,44,50,41,50,58,49,45,47,48,43,48,46,39,49,44,54,51,49,42,48,46,47,52,47,43,51,48,50,47,46,58,63,48,49,49,52,58,47,50,46,50,50,50,51,46,50,48,54,45,59,42,60,52,48,43,55,46,49,48,60,54,45,50,46,48,47,51,46,53,47,46,43,55,54,52,44,44,53,47,55,43,46,51,50,42,51,52,50,41,44,46,46,60,44,57,47,44,49,49,50,47,56,44,51,56,48,46,51,55,48,52,48,47,50,56,45,56,43,55,43,50,46,45,48,48,53,54,51,54,51,52,51,42,40,49,48,48,49,54,50,51,51,51,46,52,49,47,46,48,47,51,55,55,49,56,51,50,50,48,54,45,45,54,43,45,48,48,45,56],[185,173,177,168,166,177,163,177,177,170,176,178,175,168,168,173,180,171,164,159,177,162,165,169,170,177,170,176,180,170,175,176,179,164,158,177,171,170,175,171,165,170,171,183,171,181,169,170,178,169,167,167,164,164,166,165,174,182,161,174,168,165,178,159,174,164,169,178,167,170,174,170,177,170,181,168,169,175,166,169,171,177,176,169,165,168,176,177,168,179,166,170,176,162,169,175,173,168,170,173,175,164,176,185,190,172,185,175,176,168,177,199,186,188,189,181,174,178,187,183,169,177,191,175,174,193,180,170,187,197,184,185,178,179,171,174,167,189,177,170,181,188,182,179,175,184,180,189,172,172,174,190,181,176,177,191,170,180,184,187,189,171,190,185,185,184,187,190,182,173,192,177,191,172,174,175,182,178,188,183,173,173,170,176,175,186,188,186,192,181,180,178,174,171,164,178,184,176,185,179],[130.1,128.3,129.7,130.5,132.5,125.3,132.3,136,126.7,124.1,134.2,125.9,127.5,128.4,132.7,132.5,135.9,135.7,140,124.4,132.2,128.6,126.9,135.2,125.5,139.5,136.1,130.3,136.4,135,128.7,123.1,134.5,124.2,127.8,131.3,131.4,127.6,128.1,127.6,124.9,126.4,126.6,120.9,126.5,131.9,129.5,132.9,129.1,126.8,128,121.3,134.3,136.2,128.1,127.4,128,132.4,124.8,133.4,129,138.9,141.8,130.1,129.2,131.8,133.7,139.6,126.8,135.3,131,132.5,130.7,130.9,127.3,131.5,129,131.8,137,132.2,138.2,126.2,138.4,133.3,128.7,125.7,134.7,132.1,134.1,127.9,138.2,137.7,126,132.7,126.3,124.6,126.9,128.3,132.8,134,124.7,126.1,128.4,129.3,126,127.8,130.2,121.2,125.8,121.8,129.1,127.3,127.6,127.8,130,124.9,128.5,126.3,128.1,125.7,126.7,124.5,122.5,132.5,121,128.3,123.7,128.1,128.5,123.5,123.3,126.3,134.8,123.8,134.5,130.1,127.1,121,125.6,128.1,125.5,133.9,128.4,124,129.7,130.4,128.8,133.4,129.3,130.5,129.7,129.9,122.7,124.9,123.5,122.4,123.6,129.4,128.2,130.4,128.6,131.1,131.3,121.9,134.5,129.7,123.8,122.4,128.4,129.8,125.5,129.3,122.7,126.2,129.7,128.6,127,127.6,130.8,122.7,126,128.8,127.7,131.4,123.2,121.8,123.5,127.6,127.7,129.8,127.7,121.6,128.9,126.8,126.1,128.8,127.6,130.4,124.8,132.4]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th>id<\/th>\n <th>sex<\/th>\n <th>age<\/th>\n <th>height<\/th>\n <th>blood_pressure<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":6,"columnDefs":[{"className":"dt-right","targets":[0,2,3,4]},{"name":"id","targets":0},{"name":"sex","targets":1},{"name":"age","targets":2},{"name":"height","targets":3},{"name":"blood_pressure","targets":4}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[6,10,25,50,100]}},"evals":[],"jsHooks":[]}</script> --- ### Data visualization .panelset[ .panel[.panel-name[blood pressure] <img src="data:image/png;base64,#lecture-1_files/figure-html/hist-bp-1.png" width="504" style="display: block; margin: auto;" /> ] .panel[.panel-name[Age] <img src="data:image/png;base64,#lecture-1_files/figure-html/hist-age-1.png" width="504" style="display: block; margin: auto;" /> ] .panel[.panel-name[Scatter plot] <img src="data:image/png;base64,#lecture-1_files/figure-html/scater-bp-age-1.png" width="504" style="display: block; margin: auto;" /> ] ] --- ### Blood pressure vs age - From the scatter plot it seems like a linear function would be a good starting point to study the association between the variables of interest. -- - The simple linear model can be expressed as: `$$Y_i = \beta_0 + \beta_1 X_{i1} + \epsilon_i$$` --- ### Notation: `$$Y_i = \beta_0 + \beta_1 X_{i1} + \epsilon_i$$` - `\(i = \{1, 2, \dots, n\}\)` is an indicator variable for the observation number. -- - `\(Y_i\)` is the **i-th** observation or outcome. -- - `\(X_{i1}\)` is the predictor for the **i-th** observation. -- - `\(\beta_0\)` is the intercept or the expected value of the outcome variable for an observation with a predictor with a value of 0. -- - `\(\beta_1\)` is the slope or the rate of change of the expected value of the outcome for a one unit difference in the predictor. --- ### Linear regression - The simple linear model has two components: -- 1. Systematic component (**aka** mean model) : `$$\mu_i = \beta_0 + \beta_1 X_{i1}$$` -- 1. Random component (**aka** error term): `$$\epsilon_i$$` -- - For now we will assume that: `$$\epsilon_i \overset{iid}{\sim} N(0,\sigma^2)$$` -- - This will make things easier for now, however, we will see that this assumption can be bent a little. --- ### What now? - With our current specification we should be able to express a participant's blood pressure as: `$$\text{blood pressure}_i = \beta_0 + \beta_1 \text{age}_{i} + \epsilon_i$$` -- - Notice that we don't know the values of `\(\beta_0\)`, `\(\beta_1\)` or `\(\epsilon_i\)`. If we had access to the entire population this would be easy, we just need to measure everyone's blood pressure and age in order to find those values. -- - This is very hard to do so instead of measuring the entire population we can try to estimate those unknown values known as *parameters* from a sample of the population instead. --- ### Parameter estimation - How can we estimate the parameters of a model? -- - We can think of estimation as a decision problem were we want to choose one or more values from a set of candidates. -- - For example, we could start by guessing that the values could be: `\(\beta_0 = 100, \quad \beta_1 = 0.5, \quad \sigma^2 = 2\)` -- - We can compare this first attempt with our data --- ### Example line `$$\mathrm{E}\left(\text{blood pressure}_i\right) = 100 + 1/2\ \text{age}_{i}$$` <img src="data:image/png;base64,#lecture-1_files/figure-html/example-line-1.png" width="504" style="display: block; margin: auto;" /> --- ## Example line - The previous plot showed us that a lot of our observations were above the line so what can we do next? -- - We can frame the problem of estimating the parameters of a model as a decision problem! -- - In this case, we want to find the values of the parameters that will let us optimize some criterion or function, which will depend on our candidate values of each of the parameters in the model. -- - We refer to these functions as **objective functions**. -- **Note:** The derivative of these objective functions also has a name: **score function** --- ### Objective functions - There are many objective functions that we could choose from (eventually we will use different ones during this course). -- - For now we will use what is known as the Mean Squared Error as an objective function. -- - According to this criterion, we want to find the values of `\(\hat{\beta}_0\)`, and `\(\hat{\beta}_1\)` that minimize the MSE. `$$\underset{\hat{\beta}_0, \hat{\beta}_1}{\operatorname{min}} = \frac{1}{n}\sum_{i = 1}^{n} \left(y_i - \hat{\beta}_0 - \hat{\beta}_1x_{i1}\right)^2$$` -- - This objective function has an intuitive interpretation and visualization. --- ### Visualization of MSE <iframe width="1200" height="470" src="https://www.geogebra.org/m/XUkhCJRj"></iframe> --- ### Solution: Normal equations - We call the solution to of the optimization problem the Ordinary Least Squares Estimators or **OLS** for short. -- - The OLS for the simple linear regression can be expressed as: `$$\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}_1$$` `$$\hat{\beta}_1 = \sum_{i = 1}^n \frac{(y_i - \bar{y})(x_{i1} - \bar{x}_1)}{(x_{i1}-\bar{x}_1)^2}$$` --- ### Estimation of the variance - To estimate the variance we use the unbiased estimator which is: `$$\hat{\sigma}^2 = \frac{1}{n-p} \sum_{i = 1}^n (y_i - \hat{\mu}_i)^2$$` -- - were `\(p\)` is the number of "betas" in the model and `$$\hat{\mu}_i = \hat{\beta}_0 + \hat{\beta}_1x_{i1}$$` --- class: center, middle, inverse ## Now it's your turn!